1. Field of the Invention
This invention relates to a correlation computing device which determines correlation between image data from, for example, two successive television image fields (or frames) on the basis of an absolute difference computing operation between the image data.
2. Description of the Prior Art
An image motion vector detecting apparatus using a prior art correlation computing device is disclosed in, for example, JP-A-61-269475.
FIG. 7 shows schematically the structure of the disclosed image motion vector detecting apparatus using the prior art correlation computing device in which an absolute difference between image data is computed so as to determine correlation between an image picked up at one moment and that picked up at the next moment. Referring to FIG. 7, the disclosed image motion vector detecting apparatus comprises a first latch 1, a representative point preservation memory 2, a second latch 3, a subtractor 4 (which may be replaced by an adder), an address controller 5, an absolute value conversion circuit 6, a cumulative adder 7, and a minimum-value address decision 8. An image input signal is applied to the latch 1 and also to the subtractor 4, and a motion vector output signal appears from the minimum-value address decision circuit 8. Timing signals LP1 and LP2 are externally applied to the first latch 1 and the second latch 3, respectively. A write/read control signal W/R is applied from the address controller 5 to the representative point preservation memory 2 to control image data writing and reading. The address controller 5 applies also a write address signal WA and a read address signal RA to the memory 2 for the purpose of address setting. The address controller 5 applies also a control signal to the cumulative adder 7.
The image motion vector detecting apparatus using the prior art correlation computing device detects an image motion vector in a manner which will be described with reference to FIGS. 8A, 8B, 8C and FIGS. 9A, 9B.
First, what is called an image motion vector will be described with reference to FIGS. 8A, 8B and 8C.
FIG. 8A shows an image of an object picked up at one moment by means of an image pick-up device such as a TV camera. FIG. 8B shows the same image after a period of one field or one frame. That is, the image makes a translational displacement due to the movement of the image pick-up device. In FIG. 8C, the amount of the translational displacement of the image is indicated by the arrowed line segment, and a vector representation of the above displacement of the image is called an image motion vector.
A so-called representative point matching method is most commonly employed so as to detect an image motion vector. FIG. 9A shows a plurality of representative points (9 points are illustrated) and the respective motion vector detecting regions selected in an image field (or frame) according to the representative point matching method. FIG. 9B shows picture elements distributed around the representative point in one of the motion vector detecting regions.
According to the representative point matching method, the image motion vector is detected by finding, with the correlation calculation between the previous and the present fields (or frames), the location in the present field (or frame) to which an image has moved from the location of the representative point in the previous field (or frame) the image having been located.
Referring to FIGS. 9A and 9B, a further detailed description of the representative point matching method will be given.
First, an absolute difference between the image data S.sub.n-1 (X.sub.k, Y.sub.k) of the representative point at a coordinate (X.sub.k, Y.sub.k) in the (n-1)th field (or frame) and the image data S.sub.n (X.sub.k+i, Y.sub.k+j) of a picture element at a coordinate (X.sub.k+i, Y.sub.k+j) around the representative point within k-th moving vector detecting region in the n-th field (or frame) is calculated, where k is the serial number for appointing a representative point, i and j are coordinate differences between the representative point and a picture element of attention, where the variable ranges of i and j correspond to the range within which a motion vector is detected, and n denotes discrete time. The above absolute difference calculation is performed between the representative point and each of the surrounding picture elements in the respective motion vector detecting regions. Then, the absolute differences, calculated for each picture element at the location having the same coordinate values on the relative coordinates fixed to the representative points in the respective moving vector detecting regions, are added together. In the case of FIG. 9A, as there are 9 representative points, the above cumulative addition is carried out for 9 absolute difference data on each picture element. This cumulative addition operation results in a set of correlations P (i, j), the number of which is equal to the number of picture elements included in one motion vector detecting region.
The arithmetic operations described above can be written in the following formula: EQU P(i, j)=.SIGMA..vertline.S.sub.n-1 (X.sub.k, Y.sub.k)-S.sub.n (X.sub.k+ i, y.sub.k+j).vertline. (1)
where .SIGMA. indicates a summation operation within the variable range of suffix number k, which varies from 1 to the number of the representative points in one image field (or frame), 9 in the case of FIG. 9A.
The correlation values P(i, j) are thus obtained, and a small P(i, j) value indicates a high correlation, a high P(i, j) value indicates a low correlation. Since the input signals are of the two pictures the one of which has resulted from a translation displacement of the other, there is only one state in which these two pictures can be overlapped each other. The overlapping state of the two pictures is detected by finding the coordinate (i, j) where a P(i, j) has a minimum value. Then, the image motion vector V is given as the coordinate value (i, j). Accordingly, the motion vector is determined with the following operation: EQU V(i, j)=(i, j) for min {P(i, j)} (2)
where "min" shows an operator for selecting the minimum. A method for determining the motion vector with above-mentioned arithmetic operations is called the representative point matching method.
The operation of the image motion vector detecting apparatus using the prior art correlation computing device will now be described with reference to FIG. 7 and FIGS. 9A, 9B.
Image data S.sub.n-1 (X.sub.k, Y.sub.k) appearing at the k-th representative point in the (n-1)th field (or frame) is supplied to the latch 1 with the timing of the timing pulse signal LP1, and, under control of the write control signal WA, written in and stored at the corresponding addresses for respective k values in the representative point preservation memory 2. In the n-th field (or next frame), the difference between the image data S.sub.n (X.sub.k+i, Y.sub.k+j) for each picture element around the k-th representative point and the image data S.sub.n-1 (X.sub.k, Y.sub.k) stored in the representative point preservation memory 2 is computed by the subtractor 4, and the absolute value conversion circuit 6 generates the absolute difference and supplies the absolute difference data to the cumulative adder 7, by which the absolute difference data calculated for each picture element at a location having the same coordinate value (i, j) on the relative coordinates fixed to the representative points in respective motion vector detecting regions are added together. After completion of the cumulative addition of the absolute difference data for all the motion vector detecting regions, a set of cumulative absolute differences, or correlations, P(i, j) are obtained, and the minimum-value address decision circuit 8 decides the location (i, j) where the minimum value of P(i, j) is held in the cumulative adder 7. In the case of the decision of correlation on the basis of the cumulative absolute difference P(i, j), the higher the correlation, the smaller the value of P(i, j). Therefore, when the position (address) of each of the representative points is taken as the reference, the distance and direction between the representative point and the position (address) where the cumulative absolute difference P(i, j) exhibits the minimum value indicates the motion vector. The above operation is done in each field (or frame). Therefore, the function of the first latch 1 is to hold the image data for each of the representative points in one field (or frame), while the computation of the correlation in the preceding field (or frame) is being performed. Also, the function of the second latch 3 is to hold the image data for the representative points so that they can be used for computing the correlation between image data of the representative points and picture elements around them in the respective motion vector detecting regions.
However, in the prior art structure described above, a memory capacity of 12 bits per word is required in the process of cumulative addition, in case of an 8-bit data signal, and 16 representative points. Also, a word length of 8 bits is required for the image data processing in the stage before the absolute value conversion circuit 6. Thus, in order to enhance the motion vector detection accuracy, the circuit scale becomes inevitably large with the increase in the number of representative points, and this is disadvantageous for the purpose of integration of the circuit into, for example, an LSI. Further, the prior art structure has an additional problem in that the elimination of low-order bits of the input signal for decreasing the word length results in reduced accuracy of motion vector detection.